Supervised Semantic Scene Classification Based on Low-Level Clustering and Relevance Feedback

A framework for semantic-based scene classification using relevance feedback is presented. The semantic component casts the classifier within a framework of the supervised –or learning-from-examples– paradigm. Selection of suitable examples and labeling training patterns imposes a certain burden on the user that increases with the complexity of the ontology involved in the scene interpretation. The proposed framework involves an on-line clustering whose intent is to create "natural" groups of patterns extracted from the scenes. The user adds some domain knowledge by labeling a number of randomly selected samples. Relevance feedback is incorporated to reinforce the training of the classifier in a 'learning with a critic' mode. To tackle the stability/plasticity dilemma that rises in changing the clusters arrangement, an intermediate structure is used to organize the patterns into semantically meaningful groups. The framework shows promising results and alleviates some of the drawbacks present when exploiting mechanisms of partial supervision when dealing with scene classification.